Papers with multi-task models
Spurious Correlations in Cross-Topic Argument Mining (2021.starsem-1)
Copied to clipboard
| Challenge: | Recent work in cross-topic argument mining attempts to learn models that generalise across topics rather than relying on within-topic spurious correlations. |
| Approach: | They propose to use linear approximations of decision boundaries and manual feature grouping to learn models that generalise across topics rather than relying on within-topic spurious correlations. |
| Outcome: | The proposed model generalise across topics rather than relying on spurious correlations. |
Controlling Text Complexity in Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | Prior work on text complexity has focused on simplifying input text in one language, primarily English. |
| Approach: | They propose a method to align news articles written for different levels of target language proficiency. |
| Outcome: | The proposed model outperforms pipeline approaches that translate and simplify text independently. |
Eliciting and Understanding Cross-task Skills with Task-level Mixture-of-Experts (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained transformer models are capable of multitasking on diverse NLP tasks, but little is known about how multitaskability and cross-task generalization is achieved. |
| Approach: | They propose to use a transformer-based mixture-of-expert model with a router component to choose among experts dynamically and flexibly. |
| Outcome: | The proposed models improve the average performance gain (ARG) metric by 2.6% when adapting to unseen tasks, and by 5.6% in zero-shot generalization settings. |
Multi-task Learning of Negation and Speculation for Targeted Sentiment Classification (2021.naacl-main)
Copied to clipboard
| Challenge: | Currently, most work on targeted sentiment analysis is focused on improving the overall results. |
| Approach: | They propose a multi-task learning method to incorporate information from syntactic and semantic auxiliary tasks to create English-language models that are more robust to linguistic phenomena. |
| Outcome: | The proposed method improves on negation and speculation datasets but there is room for improvement. |
Lifelong Language Knowledge Distillation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to perform lifelong language learning (LLL) on stream of different tasks are challenging . Existing models face catastrophic forgetting problem, which can be mitigated by lifelong learning . |
| Approach: | They propose a method that can be easily applied to existing LLL architectures to mitigate degradation. |
| Outcome: | The proposed method improves state-of-the-art models and reduces degradation compared to multi-task models. |
Dynamic Fisher-weighted Model Merging via Bayesian Optimization (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing merging approaches involve scaling the parameters model-wise or integrating parameter importance parameter-wise. |
| Approach: | They propose a method for merging model-based models at the parameter level without training data or joint training. |
| Outcome: | The proposed model merging framework outperforms baseline models on validation sets. |
Multi-Task Learning for Japanese Predicate Argument Structure Analysis (N19-1)
Copied to clipboard
| Challenge: | Recent work ignores event-nouns or builds a single model for solving both tasks . however, there are interactions between predicates and event-nons, making it difficult to target only predicate. |
| Approach: | They propose a multi-task learning method that targets event-nouns . their results improve performance of both PASA and ENASA tasks . |
| Outcome: | The proposed model improves both PASA and ENASA tasks compared to a single-task model . it is the first work to employ neural networks in ENASA . |
Learning Task Sampling Policy for Multitask Learning (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to train multi-task models with auxiliary tasks are limited by the number of combinations and the importance of each auxiliary task is not known a priori. |
| Approach: | They propose a search method that automatically assigns importance weights to auxiliary tasks to improve the target task quality. |
| Outcome: | The proposed method outperforms uniform sampling and the corresponding single-task baseline on XNLI and GLUE. |
Multi-task Adversarial Attacks against Black-box Model with Few-shot Queries (2025.acl-long)
Copied to clipboard
| Challenge: | Existing adversarial text attacks rely on abundant access to shared internal features and numerous queries, limited to a single task type. |
| Approach: | They propose a black-box attack that exploits the transferability of adversarial texts . they use a deep-level substitute model trained in a plug-and-play manner for text classification . |
| Outcome: | The proposed attack can target multiple tasks with minimal perturbations . it can target commercial APIs, large language models, and image-generation models . |
Multilingual Generation and Answering of Questions from Texts and Knowledge Graphs (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for QG-QA are limited to English, but can be used in other languages. |
| Approach: | They propose to bring multilinguality to multimodal QG-QA by using Brazilian Portuguese and Russian data. |
| Outcome: | The proposed approach outperforms a baseline on English and can handle both languages. |